Perceptual-Similarity-Aware Deep Speaker Representation Learning for Multi-Speaker Generative Modeling

We propose novel deep speaker representation learning that considers perceptual similarity among speakers for multi-speaker generative modeling. Following its success in accurate discriminative modeling of individuality, knowledge (i.e., using neural networks) has been introduced to However, the conventional algorithm does not necessarily learn embeddings suitable such modeling, which may result lower quality and less controllability synthetic speech. three algorithms utilize a matrix obtained by large-scale scoring speaker-pair similarity. The train encoder with different representations matrix: set vectors, Gram matrix, graph. Furthermore, we an active iterates training. To obtain while reducing costs training, selects unscored speaker-pairs be scored next on basis sequentially-trained encoder's prediction results. Experimental evaluation results show 1) proposed strongly correlated similarity, 2) improve speech autoencoding tasks better than d-vectors learned 3) achieves higher 4) {vector, graph} embedding algorithms, first best third gives most improvement naturalness.